Recommendations for accelerating open preprint peer review to improve the culture of science.

Peer review is an important part of the scientific process, but traditional peer review at journals is coming under increased scrutiny for its inefficiency and lack of transparency. As preprints become more widely used and accepted, they raise the possibility of rethinking the peer-review process. Preprints are enabling new forms of peer review that have the potential to be more thorough, inclusive, and collegial than traditional journal peer review, and to thus fundamentally shift the culture of peer review toward constructive collaboration. In this Consensus View, we make a call to action to stakeholders in the community to accelerate the growing momentum of preprint sharing and provide recommendations to empower researchers to provide open and constructive peer review for preprints.

Characterization of Comments About bioRxiv and medRxiv Preprints.

Importance: Preprints have been increasingly used in biomedical science, and a key feature of many platforms is public commenting. The content of these comments, however, has not been well studied, and it is unclear whether they resemble those found in journal peer review. Objective: To describe the content of comments on the bioRxiv and medRxiv preprint platforms. Design, Setting, and Participants: In this cross-sectional study, preprints posted on the bioRxiv and medRxiv platforms in 2020 were accessed through each platform's application programming interface on March 29, 2021, and a random sample of preprints containing between 1 and 20 comments was evaluated independently by 3 evaluators using an instrument to assess their features and general content. Main Outcome and Measures: The numbers and percentages of comments from authors or nonauthors were assessed, and the comments from nonauthors were assessed for content. These nonauthor comments were assessed to determine whether they included compliments, criticisms, corrections, suggestions, or questions, as well as their topics (eg, relevance, interpretation, and methods). Nonauthor comments were also analyzed to determine whether they included references, provided a summary of the findings, or questioned the preprint's conclusions. Results: Of 52 736 preprints, 3850 (7.3%) received at least 1 comment (mean [SD] follow-up, 7.5 [3.6] months), and the 1921 assessed comments (from 1037 preprints) had a median length of 43 words (range, 1-3172 words). The criticisms, corrections, or suggestions present in 694 of 1125 comments (61.7%) were the most prevalent content, followed by compliments (n = 428 [38.0%]) and questions (n = 393 [35.0%]). Criticisms usually regarded interpretation (n = 286), methodological design (n = 267), and data collection (n = 238), while compliments were mainly about relevance (n = 111) and implications (n = 72). Conclusions and Relevance: In this cross-sectional study of preprint comments, topics commonly associated with journal peer review were frequent. However, only a small percentage of preprints posted on the bioRxiv and medRxiv platforms in 2020 received comments on these platforms. A clearer taxonomy of peer review roles would help to describe whether postpublication peer review fulfills them.

Are most published research findings false in a continuous universe?

Diagnostic screening models for the interpretation of null hypothesis significance test (NHST) results have been influential in highlighting the effect of selective publication on the reproducibility of the published literature, leading to John Ioannidis' much-cited claim that most published research findings are false. These models, however, are typically based on the assumption that hypotheses are dichotomously true or false, without considering that effect sizes for different hypotheses are not the same. To address this limitation, we develop a simulation model that overcomes this by modeling effect sizes explicitly using different continuous distributions, while retaining other aspects of previous models such as publication bias and the pursuit of statistical significance. Our results show that the combination of selective publication, bias, low statistical power and unlikely hypotheses consistently leads to high proportions of false positives, irrespective of the effect size distribution assumed. Using continuous effect sizes also allows us to evaluate the degree of effect size overestimation and prevalence of estimates with the wrong sign in the literature, showing that the same factors that drive false-positive results also lead to errors in estimating effect size direction and magnitude. Nevertheless, the relative influence of these factors on different metrics varies depending on the distribution assumed for effect sizes. The model is made available as an R ShinyApp interface, allowing one to explore features of the literature in various scenarios.

Recommendations for empowering early career researchers to improve research culture and practice.

Early career researchers (ECRs) are important stakeholders leading efforts to catalyze systemic change in research culture and practice. Here, we summarize the outputs from a virtual unconventional conference (unconference), which brought together 54 invited experts from 20 countries with extensive experience in ECR initiatives designed to improve the culture and practice of science. Together, we drafted 2 sets of recommendations for (1) ECRs directly involved in initiatives or activities to change research culture and practice; and (2) stakeholders who wish to support ECRs in these efforts. Importantly, these points apply to ECRs working to promote change on a systemic level, not only those improving aspects of their own work. In both sets of recommendations, we underline the importance of incentivizing and providing time and resources for systems-level science improvement activities, including ECRs in organizational decision-making processes, and working to dismantle structural barriers to participation for marginalized groups. We further highlight obstacles that ECRs face when working to promote reform, as well as proposed solutions and examples of current best practices. The abstract and recommendations for stakeholders are available in Dutch, German, Greek (abstract only), Italian, Japanese, Polish, Portuguese, Spanish, and Serbian.

High associative neuron numbers could drive cognitive performance in corvid species.

Corvids possess cognitive skills, matching those of nonhuman primates. However, how these species with their small brains achieve such feats remains elusive. Recent studies suggest that cognitive capabilities could be based on the total numbers of telencephalic neurons. Here we extend this hypothesis further and posit that especially high neuron counts in associative pallial areas drive flexible, complex cognition. If true, avian species like corvids should specifically accumulate neurons in the avian associative areas meso- and nidopallium. To test the hypothesis, we analyzed the neuronal composition of telencephalic areas in corvids and noncorvids (chicken, pigeons, and ostriches-the species with the largest bird brain). The overall number of pallial neurons in corvids was much higher than in chicken and pigeons and comparable to those of ostriches. However, neuron numbers in the associative mesopallium and nidopallium were twice as high in corvids and, in correlation with these associative areas, the corvid subpallium also contained high neuron numbers. These findings support our hypothesis that large absolute numbers of associative pallial neurons contribute to cognitive flexibility and complexity and are key to explain why crows are smart. Since meso-/nidopallial and subpallial areas scale jointly, it is conceivable that associative pallio-striatal loops play a similar role in executive decision making as described in primates.

Iniciativa Brasileira de Reprodutibilidade

A Iniciativa Brasileira de Reprodutibilidade é uma iniciativa multicêntrica para estimar a reprodutibilidade da ciência biomédica brasileira. Financiada pelo Instituto Serrapilheira, nossa meta é reproduzir uma amostra de 60 a 80 experimentos de artigos brasileiros, inicialmente em 3 técnicas diferentes: ensaio de viabilidade celular por MTT, PCR quantitativo - RT-PCR e labirinto elevado em cruz). O sitio web traz informação sobre publicações, atividades, parcerias, eventos e outros projetos do grupo

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science.

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.

Using mixed methods to construct and analyze a participatory agent-based model of a complex Zimbabwean agro-pastoral system.

Complex social-ecological systems can be difficult to study and manage. Simulation models can facilitate exploration of system behavior under novel conditions, and participatory modeling can involve stakeholders in developing appropriate management processes. Participatory modeling already typically involves qualitative structural validation of models with stakeholders, but with increased data and more sophisticated models, quantitative behavioral validation may be possible as well. In this study, we created a novel agent-based-model applied to a specific context: Zimbabwean non-governmental organization the Muonde Trust has been collecting data on their agro-pastoral system for the last 35 years and had concerns about land-use planning and the effectiveness of management interventions in the face of climate change. We collaboratively created an agent-based model of their system using their data archive, qualitatively calibrating it to the observed behavior of the real system without tuning any parameters to match specific quantitative outputs. We then behaviorally validated the model using quantitative community-based data and conducted a sensitivity analysis to determine the relative impact of underlying parameter assumptions, Indigenous management interventions, and different rainfall variation scenarios. We found that our process resulted in a model which was successfully structurally validated and sufficiently realistic to be useful for Muonde researchers as a discussion tool. The model was inconsistently behaviorally validated, however, with some model variables matching field data better than others. We observed increased model system instability due to increasing variability in underlying drivers (rainfall), and also due to management interventions that broke feedbacks between the components of the system. Interventions that smoothed year-to-year variation rather than exaggerating it tended to improve sustainability. The Muonde trust has used the model to successfully advocate to local leaders for changes in land-use planning policy that will increase the sustainability of their system.

The relationship between the number of neurons and behavioral performance in Swiss mice.

Neuronal number varies by several orders of magnitude across species, and has been proposed to predict cognitive capability across species. Remarkably, numbers of neurons vary across individual mice by a factor of 2 or more. We directly addressed the question of whether there is a relationship between performance in behavioral tests and the number of neurons in functionally relevant structures in the mouse brain. Naïve Swiss mice went through a battery of behavioral tasks designed to measure cognitive, motor and olfactory skills. We estimated the number of neurons in different brain regions (cerebral cortex, hippocampus, olfactory bulb, cerebellum and remaining areas) and crossed the two datasets to test the a priori hypothesis of correlation between cognitive abilities and numbers of neurons. Surprisingly, performance in the behavioral tasks did not correlate strongly with number of neurons in any of the brain regions studied. Our results show that whereas neuronal number is a predictor of cognitive skills across species, it is not a good predictor of cognitive, sensory or motor ability across individuals within a species, which suggests that other factors are more relevant for explaining cognitive differences between individuals of the same species.

Addressing selective reporting of experiments through predefined exclusion criteria.

The pressure for every research article to tell a clear story often leads researchers in the life sciences to exclude experiments that 'did not work' when they write up their results. However, this practice can lead to reporting bias if the decisions about which experiments to exclude are taken after data have been collected and analyzed. Here we discuss how to balance clarity and thoroughness when reporting the results of research, and suggest that predefining the criteria for excluding experiments might help researchers to achieve this balance.

Two years into the Brazilian Reproducibility Initiative: reflections on conducting a large-scale replication of Brazilian biomedical science

Scientists have increasingly recognised that low methodological and analytical rigour combined with publish-or-perish incentives can make the published scientific literature unreliable. As a response to this, large-scale systematic replications of the literature have emerged as a way to assess the problem empirically. The Brazilian Reproducibility Initiative is one such effort, aimed at estimating the reproducibility of Brazilian biomedical research. Its goal is to perform multicentre replications of a quasi-random sample of at least 60 experiments from Brazilian articles published over a 20-year period, using a set of common laboratory methods. In this article, we describe the challenges of managing a multicentre project with collaborating teams across the country, as well as its successes and failures over the first two years. We end with a brief discussion of the Initiative's current status and its possible future contributions after the project is concluded in 2021.

The reliability of the isotropic fractionator method for counting total cells and neurons.

BACKGROUND: The Isotropic Fractionator (IF) is a method to determine the cellular composition of nervous tissue. It has been mostly applied to assess variation across species, where differences are expected to be large enough not to be masked by methodological error. However, understanding the sources of variation in the method is important if the goal is to detect smaller differences, for example, in same-species comparisons. Comparisons between different mice strains suggest that the IF is consistent enough to detect these differences. Nevertheless, the reliability of the method has not yet been examined directly. METHOD: In this study, we evaluate the reliability of the method for the determination of cellular and neuronal numbers of Swiss mice. We performed repeated cell counts of the same material by different experimenters to quantify different sources of variation. RESULTS: In total cell counts, we observed that for the cerebral cortex most of the variance was at the counter level. For the cerebellum, most of the variance is attributed to the sample itself. As for neurons, random error along with the immunostaining correspond to most of the variation, both in the cerebral cortex and in the cerebellum. Test-retest reliability coefficients were relatively high, especially for cell counts. CONCLUSIONS: Although biases between counters and random variation in staining could be problematic when aggregating data from different sources, we offer practical suggestions to improve the reliability of the method. While small, this study is a most needed step towards more precise measurement of the brain's cellular composition.

White matter volume and white/gray matter ratio in mammalian species as a consequence of the universal scaling of cortical folding.

Because the white matter of the cerebral cortex contains axons that connect distant neurons in the cortical gray matter, the relationship between the volumes of the 2 cortical compartments is key for information transmission in the brain. It has been suggested that the volume of the white matter scales universally as a function of the volume of the gray matter across mammalian species, as would be expected if a global principle of wiring minimization applied. Using a systematic analysis across several mammalian clades, here we show that the volume of the white matter does not scale universally with the volume of the gray matter across mammals and is not optimized for wiring minimization. Instead, the ratio between volumes of gray and white matter is universally predicted by the same equation that predicts the degree of folding of the cerebral cortex, given the clade-specific scaling of cortical thickness, such that the volume of the gray matter (or the ratio of gray to total cortical volumes) divided by the square root of cortical thickness is a universal function of total cortical volume, regardless of the number of cortical neurons. Thus, the very mechanism that we propose to generate cortical folding also results in compactness of the white matter to a predictable degree across a wide variety of mammalian species.

The Brazilian Reproducibility Initiative.

Most efforts to estimate the reproducibility of published findings have focused on specific areas of research, even though science is usually assessed and funded on a regional or national basis. Here we describe a project to assess the reproducibility of findings in biomedical science published by researchers based in Brazil. The Brazilian Reproducibility Initiative is a systematic, multicenter effort to repeat between 60 and 100 experiments: the project will focus on a set of common methods, repeating each experiment in three different laboratories from a countrywide network. The results, due in 2021, will allow us to estimate the level of reproducibility of biomedical science in Brazil, and to investigate what aspects of the published literature might help to predict whether a finding is reproducible.

No relative expansion of the number of prefrontal neurons in primate and human evolution.

Human evolution is widely thought to have involved a particular expansion of prefrontal cortex. This popular notion has recently been challenged, although controversies remain. Here we show that the prefrontal region of both human and nonhuman primates holds about 8% of cortical neurons, with no clear difference across humans and other primates in the distribution of cortical neurons or white matter cells along the anteroposterior axis. Further, we find that the volumes of human prefrontal gray and white matter match the expected volumes for the number of neurons in the gray matter and for the number of other cells in the white matter compared with other primate species. These results indicate that prefrontal cortical expansion in human evolution happened along the same allometric trajectory as for other primate species, without modification of the distribution of neurons across its surface or of the volume of the underlying white matter. We thus propose that the most distinctive feature of the human prefrontal cortex is its absolute number of neurons, not its relative volume.

The elephant brain in numbers.

What explains the superior cognitive abilities of the human brain compared to other, larger brains? Here we investigate the possibility that the human brain has a larger number of neurons than even larger brains by determining the cellular composition of the brain of the African elephant. We find that the African elephant brain, which is about three times larger than the human brain, contains 257 billion (10(9)) neurons, three times more than the average human brain; however, 97.5% of the neurons in the elephant brain (251 billion) are found in the cerebellum. This makes the elephant an outlier in regard to the number of cerebellar neurons compared to other mammals, which might be related to sensorimotor specializations. In contrast, the elephant cerebral cortex, which has twice the mass of the human cerebral cortex, holds only 5.6 billion neurons, about one third of the number of neurons found in the human cerebral cortex. This finding supports the hypothesis that the larger absolute number of neurons in the human cerebral cortex (but not in the whole brain) is correlated with the superior cognitive abilities of humans compared to elephants and other large-brained mammals.

Cellular scaling rules for the brain of afrotherians.

Quantitative analysis of the cellular composition of rodent, primate and eulipotyphlan brains has shown that non-neuronal scaling rules are similar across these mammalian orders that diverged about 95 million years ago, and therefore appear to be conserved in evolution, while neuronal scaling rules appear to be free to vary in evolution in a clade-specific manner. Here we analyze the cellular scaling rules that apply to the brain of afrotherians, believed to be the first clade to radiate from the common eutherian ancestor. We find that afrotherians share non-neuronal scaling rules with rodents, primates and eulipotyphlans, as well as the coordinated scaling of numbers of neurons in the cerebral cortex and cerebellum. Afrotherians share with rodents and eulipotyphlans, but not with primates, the scaling of number of neurons in the cortex and in the cerebellum as a function of the number of neurons in the rest of the brain. Afrotheria also share with rodents and eulipotyphlans the neuronal scaling rules that apply to the cerebral cortex. Afrotherians share with rodents, but not with eulipotyphlans nor primates, the neuronal scaling rules that apply to the cerebellum. Importantly, the scaling of the folding index of the cerebral cortex with the number of neurons in the cerebral cortex is not shared by either afrotherians, rodents, or primates. The sharing of some neuronal scaling rules between afrotherians and rodents, and of some additional features with eulipotyphlans and primates, raise the interesting possibility that these shared characteristics applied to the common eutherian ancestor. In turn, the clade-specific characteristics that relate to the distribution of neurons along the surface of the cerebral cortex and to its degree of gyrification suggest that these characteristics compose an evolutionarily plastic suite of features that may have defined and distinguished mammalian groups in evolution.